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Abstract 

The  reduced  basis  method  was  introduced  for  the  accurate  online  evaluation  of 
solutions  to  a  parameter  dependent  family  of  elliptic  partial  differential  equations. 
Abstractly,  it  can  be  viewed  as  determining  a  “good”  n  dimensional  space  7in  to  be 
used  in  approximating  the  elements  of  a  compact  set  T  in  a  Hilbert  space  7 i.  One,  by 
now  popular,  computational  approach  is  to  find  TLn  through  a  greedy  strategy.  It  is 
natural  to  compare  the  approximation  performance  of  the  7in  generated  by  this  strategy 
with  that  of  the  Kolmogorov  widths  dn(J-)  since  the  latter  gives  the  smallest  error  that 
can  be  achieved  by  subspaces  of  fixed  dimension  to.  The  first  such  comparisons,  given  in 
[1],  show  that  the  approximation  error,  an(!F)  :=  dist(V, 7in),  obtained  by  the  greedy 
strategy  satisfies  <rn(.F)  <  Cn2ndn(T).  In  this  paper,  various  improvements  of  this 
result  will  be  given.  Among  these,  it  is  shown  that  whenever  dn{J-)  <  Mn_Q,  for  all 
n  >  0,  and  some  M,a  >  0,  we  also  have  crn(V)  <  CaMn~a  for  all  n  >  0,  where 
Ca  depends  only  on  a.  Similar  results  are  derived  for  generalized  exponential  rates  of 
the  form  Me-®"”.  The  exact  greedy  algorithm  is  not  always  computationally  feasible 
and  a  commonly  used  computationally  friendly  variant  can  be  formulated  as  a  “weak 
greedy  algorithm”.  The  results  of  this  paper  are  established  for  this  version  as  well. 


1  Introduction 

1.1  The  reduced  basis  method 

Many  design  problems  in  engineering  can  be  formulated  as  optimal  control  problems  with 
parametric  PDE  constraints.  Typically,  solving  these  control  problems  requires  the  frequent 
numerical  solution  of  a  PDE  depending  on  dynamically  updated  parameters.  A  model  situa¬ 
tion  that  should  serve  as  a  guiding  example,  may  be  sketched  as  follows.  Let  77  be  a  Hilbert 

*This  research  was  supported  by  the  Fondation  Sciences  Mathematiques  de  Paris,  the  Office  of  Naval  Re¬ 
search  Contracts  ONR-N00014-07-1-0978  ONR-N00014-08- 1-1113,  and  ONR  N00014-09-1-0107;  the  AFOSR 
Contract  FA95500910500;  the  ARO/DoD  Contracts  W911NF-05-1-0227  and  W911NF-07-1-0185;  the  NSF 
Grants  DMS-0721621,  DMS-0810869,  DMS-0915104,  and  DMS  0915231;  the  Special  Priority  Program  SPP 
1324  funded  by  DFG;  and  the  Polish  MNiSW  grant  N201  269335. 
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space  with  norm  ||  •  ||  and  let  (aM)Me£>  be  a  family  of  symmetric,  continuous  and  elliptic 
bilinear  forms 

:  TL  x  Tt  — »  M, 

depending  on  parameters  p  G  D,  where  D  is  a  compact  set  in  Md,  such  that  all  the  norms 
||  •  1 1 ii  :=a,,(-,')b2  are  uniformly  equivalent  to  ||  •  ||,  i.e. 

c*||w||  <  11^11^  <  C*||v||,  v  G  7i, 

holds  for  some  constants  0  <  c*  <  C*  independent  of  fi.  For  a  given  linear  functional  L  G 
we  denote  by  u ^  the  solution  of  the  variational  problem 

a^u^v)  =  L(v),  veH.  (1.1) 

A  typical  problem  is  to  control  a  functional  quantity  J{u through  the  parameter  /i.  For 
example,  one  may  wish  to  drive  the  values  £( u towards  some  target  value  £*,  where  £  G  W 
is  a  given  linear  functional.  If  such  an  optimization  is  to  be  performed  online,  it  would 
be  infeasible  to  employ  a  standard  highly  accurate  numerical  solution  of  the  PDE  for  each 
parameter  value  ft  G  D,  arising  e.g.  in  the  course  of  a  descent  procedure. 

The  reduced  basis  method  (see  [1,  3,  4,  5,  6,  8])  has  been  introduced  as  a  strategy  to  deal 
with  such  tasks  by  exploiting  the  fact  that  the  set 

-A  :=  {u^  :  p  G  D} 

of  all  parameter  dependent  solutions  is  compact  in  the  energy  space  7i.  This  allows  a 
model  reduction  along  the  following  lines.  One  determines  first  offline  (possibly  at  high 
computational  cost)  a  fixed  subspace  7 in  of  the  energy  space  (of  possibly  small  dimension 
n )  from  which  any  element  in  the  compact  set  T  can  be  well  approximated.  For  instance, 
a  low  dimensional  Galerkin  scheme  based  on  7in  could  be  used  for  rapidly  computing  an 
approximate  solution  for  any  parameter  query.  The  subspace  itself  is  often  generated  as  the 
span  of  solutions  fj  :=  ultj ,  j  —  0, . . . ,  n  —  1,  to  specific  parameters  /xy ,  forming  the  reduced 
basis. 

A  key  question  therefore  is  to  End  “good”  parameters  /xy ,  or  equivalently,  good  basis 
functions  fj  G  T  in  the  preprocessing  step.  Here  “good”  means  that  relatively  few  basis 
functions  suffice  in  order  to  warrant  a  desired  accuracy  for  any  parameter  query. 


1.2  A  Greedy  Algorithm 

One  by  now  prominent  approach  to  the  problem  of  finding  a  good  parameter  selection  in 
the  above  setting  is  to  employ  a  greedy  strategy,  see  e.g  [1,  3,  4,  5,  8].  For  conceptual  clarity 
we  shall  describe  first  a  somewhat  idealized  version  in  the  following  abstract  setting.  We 
are  given  a  compact  set  T  in  a  Hilbert  space  7i.  The  problem  is  then  to  hnd  functions 
{/o,  /i,  •  •  • ,  fn-i}  so  that  each  /  G  T  is  well  approximated  by  the  elements  of  the  subspace 
Fn  \=  span{/0, . . . , 
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The  greedy  algorithm  for  generating  fo,  fi, .  ■  ■  proceeds  as  follows.  First,  one  chooses  a 
function  /0  such  that  ||/o||  =  max/ejr  ||/||,  where  again  here  and  later  ||  ■  ||  is  the  norm  on  H 
induced  by  its  inner  product  (-,  •).  We  define 


a0  ■■=  a0(F)  :=  ||  f0 


The  space  F\  :=  span{/0}  is  the  first  subspace  used  to  approximate  the  elements  of  F .  At 
the  general  step,  if  /0,  A,  -  -  - ,  fn- 1  have  been  chosen,  Fn  :=  span {/0,  /i, . . . ,  fn- 1}  and  Pn  is 
the  projector  onto  Fn,  then  for  any  /  G  F,  we  denote  by 

an(f)  :=||/-P„/|| 

its  error  of  best  approximation  in  Fn.  We  define 

crn  :=  an(F)  :=  ma xan(f), 
f  eV" 

which  is  the  error  in  approximating  the  set  F  using  the  elements  of  Fn.  We  now  choose  fn 
as 

fn  :=  argmax an(f),  (1.2) 

fer 

as  the  next  element  in  the  greedy  selection.  Notice  that  the  sequence  /o,  fi,  ■  ■  ■  is  generally 
not  unique  and  neither  is  the  sequence  ( crn(F))n>o  since  it  depends  on  this  selection. 

We  emphasize  that  this  greedy  algorithm  differs  from  those  which  aim  at  approximating 
one  given  function  /  by  a  finite  linear  combination  of  elements  from  a  given  dictionnary 
(see  [7]  for  a  complete  survey  on  this  type  of  greedy  algorithms).  In  the  algorithm  described 
above  the  objective  is  to  simultaneously  approximate  all  the  functions  in  the  class  F  using 
finite  dimensional  subspaces  generated  from  elements  of  F . 

The  space  Fn  is  an  n- dimensional  subspace  of  H  and  to  assess  its  quality  it  is  natural 
to  compare  this  subspace  with  the  best  n-dimensional  subspace  that  would  minimize  the 
projection  error  for  the  whole  class  F  over  all  n-dimensional  subspaces.  This  minimal  error 
is  given  by  the  Kolmogorov  width 

dn(F)  :=  inf  sup  dist (/,  K),  (1.3) 

dim(Y)=n 

where  dist (f,Y)  :=  minsey  \\f-g\\  =  || /  —  Pyf  ||  with  Py  denoting  the  orthogonal  projector 
onto  Y,  and  where  the  infimum  is  taken  over  all  n  dimensional  subspaces  Y  of  PL.  We  refer 
to  [2]  for  a  general  discussion  on  Kolmogorov  width.  We  also  define 

d0(F)  :=  max  ll/ll  =  a0(F), 

which  corresponds  to  approximating  by  zero  dimensional  spaces. 

Of  course,  if  an{F)  decayed  at  a  rate  comparable  to  dn(F),  this  would  mean  that  the 
greedy  selection  provides  essentially  the  best  possible  accuracy  attainable  by  n-dimensional 
subspaces.  The  central  objective  of  this  paper  is  to  establish  estimates  for  the  sequence 
an{F)  in  terms  of  the  widths  dn{F). 
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A  startling  result  in  this  regard  was  recently  given  in  [1]  where  it  was  proved  that 

<?n{F)  <  CnTdn{T ),  (1.4) 


with  C  an  absolute  constant.  While  this  is  an  interesting  comparison,  it  is  only  useful  if 
dn(P)  decays  to  zero  faster  than  n~12~n.  On  the  other  hand,  since  by  assumption  T  is  com¬ 
pact,  we  always  have  cr n{T)  — >  0  as  n  — >  oo.  A  hrst  central  theme  in  this  paper  is  to  improve 
on  (1.4)  in  the  general  Hilbert  space  setting  regarding  the  following  central  questions:  hrst, 
is  (1.4)  sharp?  and  second,  what  can  be  said  for  the  whole  range  of  possible  decay  rates  of 
the  widths  dn(P)  between  polynomial  and  exponential  where  (1.4)  gives  no  information? 


1.3  The  Weak  Greedy  Algorithm 

Before  explaining  our  results  in  a  bit  more  detail,  let  us  pause  to  remark  that,  for  the  original 
motivation  of  the  reduced  basis  method  for  solving  parametric  PDEs,  the  execution  of  the 
algorithm  in  the  above  form  (1.2)  is  computationally  not  feasible,  even  in  offline  mode.  In 
fact,  to  solve  (1.2),  either  one  would  have  to  solve  the  PDE  for  essentially  all  parameters  in 
(a  highly  resolved  discretized)  parameter  domain  or  possibly  employ  a  descent  strategy  to 
find  this  maximum,  which  would  also  involve  a  large  number  of  such  solves. 

However,  a  variant  of  the  greedy  strategy  (1.2)  has  been  shown  to  become  practically 
feasible  in  the  context  of  (1.1)  (see  e.g.  [5])  when  replacing  the  error  \\f  —  Pnf\\  by  a  surrogate 
rn(f)  satisfying 

crrn(f)  <  ||/  -  Pnf\\  <  Crrn(f),  feH ,  (1.5) 

for  some  fixed  constants  cr,Cr.  In  fact,  in  the  context  of  (1.1),  one  can  find  surrogates 
satisfying  (1.5)  such  that 

fn  ■=  argmax f&Tr n (/) ,  (1.6) 

can  be  computed  in  a  much  more  efficient  way  (at  least  for  a  wide  class  of  bilinear  forms  aM) 
than  the  exact  greedy  step  (1.2). 

What  is  important  for  us,  is  that  the  numerically  feasible  variant  (1.6)  produces  an 
element  fneP  that  actually  satisfies 

II  fn  ~  Pnfn  ||  >  £-max\\f  -  Pnf\\, 

as  one  easily  checks,  where  cr ,  Cr  are  the  constants  from  (1.5). 

This  allows  us  to  embed  such  practical  variants  into  a  general  abstract  setting  by  consid¬ 
ering  the  following  weak  greedy  algorithm.  This  algorithm  depends  on  a  constant  0  <  7  <  1 
(in  the  above  example  7  =  cr/Cr)  which  represents  the  relative  accuracy  to  which  we  can 
compute  norms.  At  the  hrst  step  of  the  algorithm,  one  chooses  a  function  /o  such  that 

ll/oll  >  where  a0  :=  a0(F)  ■=  max  ||/||. 

J 

At  the  general  step,  if  /0, . . . ,  fn_i  have  been  chosen,  Fn  :=  span {/0, . . . ,  fn_  1},  and  Pn  is 
the  projector  onto  Fn,  then  for  any  /  G  T  we  again  denote  by 

an(f)  :=  ||/-PB/|| 
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its  best  approximation  error  onto  Fn  and  define 

crn  :=  an(F)  :=  maxcrn(/), 
f 

which  is  the  error  in  approximating  the  class  T  using  the  elements  of  Fn.  We  now  choose  fn 
as  an  element  in  T  such  that 

crn(fn)  >  7maxcrn(/),  (1.7) 

J 

as  the  next  element  in  the  greedy  selection.  Note  that  if  7  =  1,  then  the  weak  greedy 
algorithm  reduces  to  the  greedy  algorithm  that  we  have  introduced  above. 

Notice  that  similar  to  the  greedy  algorithm,  (an(F))n> 0  is  also  monotone  decreasing.  Of 
course,  the  weak  greedy  algorithm  does  not  give  a  unique  sequence  /0,/i,...,  nor  is  the 
sequence  ( crn(F))n>o  unique.  In  all  that  follows,  the  notation  reflects  any  sequences  which 
can  arise  in  the  implementation  of  the  weak  greedy  selection  for  the  fixed  value  of  7. 

Although  not  explicitly  phrased  in  the  framework  of  weak  greedy  schemes,  it  was  also 
shown  in  [1]  that,  employing  a  surrogate  as  in  (1.6),  one  obtains  an  estimate  of  the  form 

an(F)  <  Cn(3ndn(F), 

where  (3  >  2  is  a  constant  depending  on  cr,  Cr  in  (1.5).  Since  f3  >  2,  this  further  reduces  the 
range  of  decay  rates  for  which  one  obtains  any  information.  Therefore,  the  same  questions 
asked  for  the  greedy  algorithm  can  be  posed  for  the  weak  greedy  algorithm  as  well. 

1.4  Objective,  results  and  layout  of  the  paper 

Our  central  objective  is  to  quantify  the  performance  of  the  greedy  and  weak  greedy  algorithm 
by  relating  the  distances  <Jn(F)  to  the  widths  dn(F).  We  address  these  issues  in  the  general 
setting  where  7 i  is  an  arbitrary  separable  Hilbert  space  and  F  C  7i  is  any  compact  subset. 
Since  the  greedy  algorithm  can  be  viewed  as  the  particular  instance  7  =  1  of  the  weak  greedy 
algorithm,  we  formulate  most  of  our  results  in  the  setting  of  the  weak  greedy  algorithm. 

I11  §2,  we  reformulate  the  problem  in  sequence  space  and  present  a  matrix  formulation  of 
the  weak  greedy  algorithm  that  will  be  used  throughout  the  remainder  of  the  paper.  Using 
this  formulation,  we  establish  a  fundamental  lemma  that  expresses  the  following  property: 
if  un(F )  is  flat  in  the  sense  that  it  does  not  decay  more  than  a  prescribed  amount  9  between 
the  index  n  and  n  +  qm  where  q  is  a  given  constant,  then  an(F)  is  controlled  by  dm(F). 

In  §3,  we  use  this  lemma  to  compare  the  rate  of  decay  of  an{F )  and  dn(F).  Our  main 
result  shows  that  for  any  a  >  0,  whenever  dn(F)  <  Mn~a  for  all  n  >  0  then  an(F)  < 
CMn~a  for  all  n  >  0,  where  C  only  depends  on  a  and  7.  I11  the  same  spirit,  we  show 
that  the  exponential  decay  dn{F)  <  Me-0"”  with  M,  a,  a  >  0  implies  a  certain  (different) 
exponential  decay  of  an(F),  giving  a  somewhat  better  resolution  of  the  range  of  decay  rates 
slower  than  2~n.  We  show  that  these  estimates  for  <rn{F)  are  robust  in  the  following  sense: 
if  the  fj  are  computed  up  to  an  error  of  the  order  e,  we  obtain  similar  estimates  up  to 
the  addition  of  Ce.  I11  the  context  of  (1.1),  such  error  typically  results  from  the  space 
discretization  process  when  computing  the  fj,  e.g.  by  the  finite  element  method. 

As  to  directly  comparing  an(F)  with  dn(F),  note  first  that  the  optimal  subspaces  in 
(1.3)  need  not  be  spanned  by  elements  of  T .  In  fact,  we  show  in  §4  that  subspaces  spanned 
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by  elements  of  P  itself  will  in  general  not  give  rise  to  distances  that  decay  as  fast  as  the 
n- widths,  see  Theorem  4.1.  In  particular,  we  can  therefore  not  expect  any  result  of  the  form 
an(P)  <  Cdn(P).  On  the  other  hand,  we  give  for  the  exact  greedy  algorithm  (7  =  1)  an 
improvement  of  (1.4)  in  that  the  factor  n  can  be  removed,  and  we  show  that  the  resulting 
bound  cannot  be  improved. 

2  A  matrix  formulation  of  the  weak  greedy  algorithm 

In  general,  the  weak  greedy  algorithm  does  not  terminate  and  we  obtain  an  infinite  sequence 
/o,  /1,  /2,  •  •  •  •  In  order  to  have  a  consistent  notation  in  what  follows,  we  shall  define  fm  :=  0, 
m  >  N,  if  the  algorithm  terminates  at  N,  i.e.  if  =  0.  By  (/*)n> 0  we  denote  the 
orthonormal  system  obtained  from  the  fn  by  Gram-Schmidt  orthogonalization,  so  that 


n— 1 


i=0 


and  in  particular 


fi  =  Pi+  1  fi  =  Y  Oijfj,  ai,j  =  (/*>  /?*),  3  <  *■ 

1=0 

There  is  110  loss  of  generality  in  assuming  that  the  infinite  dimensional  Hilbert  space  7i  is 
^(N  U  {0})  and  that  f*  =  eq,  where  &j  is  the  vector  with  a  one  in  the  coordinate  indexed  by 
j  and  all  other  coordinates  zero,  i.e.  (ej)i  =  8J:l.  We  adhere  to  this  assumption  throughout 
the  paper. 

We  consider  the  lower  triangular  matrix 

A  -=  ( ai,j)i,j=0i  ai,j  '=  j  >  i- 

This  matrix  incorporates  all  the  information  about  the  greedy  algorithm  on  T .  The  following 
two  properties  characterize  a  lower  triangular  matrix  A  generated  by  such  a  greedy  algorithm. 
With  the  notation  an  :=  <rn(jF),  we  have 

PI:  The  diagonal  elements  of  A  satisfy  7 an  <  |ani„|  <  an. 

P2:  For  every  m>n  one  has  Y^j "=narn,j  —  an- 

Indeed,  PI  follows  from 


aln=\\fn\\2-\\Pnfn\\2=\\fn-Pnfn 


combined  with  the  weak  greedy  selection  property  (1.7).  To  see 


2 

1 

P2  we  note  that  for  m  >  n 


Y  =  11  fm-  Pnfmf  <  max  11/  -  Pnf  ||2  =  a2n. 

J 

3=n 
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Remark  2.1  If  A  is  any  matrix  satisfying  PI  and  P2  with  an  a  strictly  decreasing  sequence, 
then  the  rows  of  A  form  a  compact  subset  of  £ 2(N  U  {0}).  If  T  is  the  set  consisting  of  these 
rows,  then  one  of  the  possible  realizations  of  the  weak  greedy  algorithm  with  constant  7  will 
choose  the  rows  in  order  and  A  will  be  the  resulting  matrix. 


The  following  result  plays  a  key  role  in  our  subsequent  analysis  of  the  convergence  rate 
of  the  weak  greedy  algorithm. 

Lemma  2.2  Fix  any  value  0  <  6  <  1  and  define  q  :=  [2,y~19~1]2 .  Suppose  that  for  some 
m,  n  e  N,  we  have 

On+qm  h  0(Tn-  (2-1) 

Then, 

en(fF)  <  cpdJ^F). 


Proof:  We  consider  the  (qm  +  1)  x  (qm  +  1)  submatrix  A'  of  A  given  by  the  entries  of  A  in 
the  rows  and  columns  with  indices  in  {n, . . .  ,n  +  qm}.  We  denote  by  gj,  j  —  0, 1, ,  qm, 
the  rows  of  A' .  These  are  each  vectors  in  whose  coordinates  we  index  by  0, 1, ... ,  qm. 

Let  Y  be  an  m-dimensional  space  which  realizes  dm{IF)  and  let  Yrn  be  the  restriction  of  Y 
to  the  coordinates  in  {n,  n  +  1, . . .  n  +  qm}.  Then,  for  each  j  the  projection  yj  of  gj  onto  Ym 
satisfies 

II 9j  ~  Vj\\  <  dm  :=  dmiJ7),  j  =  0, 1, qm. 

We  can  without  loss  of  generality  assume  that  dim(Pm)  =  m.  Let  f>\, ... ,  <fm  be  an  orthonor¬ 
mal  basis  for  Ym.  Since  each  </>,  =  (0i(j))f=o,  i  =  1 , ...  ,m,  has  norm  one,  for  at  least  one 
value  of  j  e  {0, . . . ,  qm},  we  have 

m 

D*«)l2<<r‘.  (2.2) 

2=1 

We  fix  this  value  of  j  and  write  y3  =  (.9j,  This  means  that  the  j'-th  coordinate 

Pj{j)  of  yj  satisfies 


1/2  1/2 

\yj(j)\  =  |  <  (Ei^^)l2)  (5^  l<Mj)|2)  < 

2=1  2=1  2=1 

where  we  have  used  (2.2)  and  Property  P2.  From  PI,  (2.1)  and  the  definition  of  q,  we  have 

1 9j{j)  \  =  \an+j,n+j\  >  T°n+j  >  7 >  ^Q~^Crn. 

ft  follows  that 

q~^n  <  1 9j(j)  ~  VjU) I  <  II 9j  ~  Vj\\  <  dm, 

which  completes  the  proof  of  the  lemma.  □ 


3  Convergence  estimates 

In  this  section  we  use  Lemma  2.2  to  estimate  the  rate  of  decay  of  a n(J-)  for  the  weak  greedy 
algorithm  from  the  rate  of  decay  of  the  Kolmogorov  width  dn(IF). 
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3.1  Polynomial  Decay 

We  first  consider  the  case  where  dn(T)  has  polynomial  decay  and  show  that  in  such  a  case 
crn(jF)  inherits  the  same  rate  of  decay. 

Theorem  3.1  Let  0  <  7  <  1  be  the  parameter  in  the  weak  greedy  algorithm.  Suppose  that 
d0{T)  <  M  and 

dn(lF)  <  Mn~a ,  n  >  0, 
for  some  M  >  0  and  a  >  0.  Then, 

crn  =  on(lF)  <  CMn~a,  n  >  0,  (3.1) 

with  C  :=  q^(Aq)a  and  q  :=  |~2“+17_1] 2. 

Proof:  First,  we  choose  6  =  2~a.  It  follows  that  q  =  which  is  the  relation  needed 

in  Lemma  2.2.  Next,  we  define  No  :=  4 q.  Note  that  q  >  9  and  No  >  36.  Then,  the  definition 
of  C  shows  that  (3.1)  holds  for  1  <  n  <  No  since 

On  <  00  <  M  <  MN*n~a  <  CMn~a,  n  <  N0. 

We  suppose  now  that  (3.1)  does  not  hold  for  some  n  >  N0  and  draw  a  contradiction.  To 
start  with  we  let  N  be  the  smallest  integer  N  >  No  for  which  (3.1)  does  not  hold.  Then 

CMN~a  <  aN.  (3.2) 

We  define  n  as  the  smallest  integer  for  which  naN~a  >  6,  i.e.  n  :=  |~0ly/"lV]  =  \N/2~\.  We 
then  have 

an  <  CMn~a  =  C MNan~aN~a  <  O^on. 

We  now  take  this  value  of  n  and  take  m  as  the  largest  integer  such  that  n  +  qm  <  N,  i.e. 
m  :=  | fN  —  n)/q\.  We  can  apply  Lemma  2.2  and  conclude  that 

oN  <  on  <  q^dm{F)  <  q^Mm~a  =  (q* Nam-a)MN~a .  (3.3) 

Since  N  >  No  >  Aq  —  2,  it  follows  that 

4gm  =  Aq  |_(fV  —  n)/q\ 

>  4(1V  —  n  —  q  +  1) 

=  41V  —  2(2|~lV/2])  —  4g  +  4 

>  2A^  -  2  -  Aq  +  4  >  N, 


where  we  have  used  the  fact  that  if  a  and  b  are  positive  integers,  then  n|_^J  >  b  —  a  +  1  and 
a\a~\  <&  +  a  —  l.  Therefore,  we  have  ^  <  Aq.  Combining  (3.2)  with  (3.3),  we  obtain 

C  <  q^(N/m)a  <  q^(Aq)a. 


This  contradicts  the  definition  of  C. 


□ 


3.2  Exponential  Rates 

We  next  use  the  same  ideas  to  derive  bounds  when  dn{3F)  decays  to  zero  like  Me_m“  for 
some  M,a,a  >  0. 

Theorem  3.2  Let  0  <  7  <  1  be  the  parameter  in  the  weak  greedy  algorithm.  Suppose  that 

dn(F)  <  Me~ana,  n>  0, 

for  some  M,  a,  a  >  0.  Then  setting  (3  :=  one  has 

an(F)  <  CMe~cn\  n  >  0,  (3.4) 

whenever  for  any  fixed  0  <  6  <  1,  one  takes  c  :=  min{|  In  61 1 ,  (4g)_"a},  C  :=  ma  x{ecN°  ,q^}, 
q  :=  [27 _10_1]2  and  N0  :=  [(Sg)13^]  =  [(8g)"+1]. 

Proof:  The  proof  is  similar  to  that  of  Theorem  3.1.  By  our  definition  of  C  we  find  that 
(3.4)  holds  for  n  <  N0  since  we  have 

<rn  <  cr0  <  ir0ecN°  e~cn(3  <  CMe~cn\  n  <  N0. 

We  suppose  now  that  (3.4)  does  not  hold  for  some  n  >  N0  and  draw  a  contradiction.  Let 
N  >  N0  be  the  smallest  integer  for  which 

CMe~cNP  <  crN. 

We  now  let  m  be  any  positive  integer  for  which 

ec{N-qm)P  e-cN0  >  or  equivalently  c(lV/3  -  (N  -  qm f)  <  \  In  Q\.  (3.5) 

Then,  using  that  (3.4)  holds  for  N  —  qm ,  (3.5)  yields 

&N-qm  <  0  1(7N- 

Now  we  can  apply  Lemma  2.2  and  conclude  that 

cpv  <  crN-qm  <  q^d^J7)  <  q^ Me~arn,a . 

We  are  left  to  show  that  there  is  a  choice  of  m  satisfying  (3.5)  and  such  that 

q\Me~ama  <CMe~cN\  (3.6) 

since  this  will  be  the  desired  contradiction.  Taking  logarithms,  (3.6)  is  equivalent  to 

-  In  q  —  In  C  <  ama  —  cN13 .  (3.7) 

Now  to  show  that  there  is  an  m  which  satisfies  both  (3.7)  and  (3.5),  we  consider 
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Clearly  m  <  and  therefore  N  —  qm  >  N/2.  From  the  mean  value  theorem,  we  have  for 
some  £  £  (N  —  qm ,  N) 

Nl~P 

Np  ~(N~  qm)13  =  fd^qm  <  qm(d{N / 2)/3"1  <  -^—p(N/2)f3~1  <  2~f3(3  <  1. 

Thus,  (3.5)  will  be  satisfied  for  the  definition  of  c.  Now  let  us  check  (3.7).  We  first  remark 
that  since  N  >  N0,  we  have  m  >  4  and  therefore  m  >  N\qP  ■  From  the  definition  of  c,  we 
thus  have 

ama  —  cN13  >  a  (  — — J  —  cN13  =  (a(4g)_a  —  c)N(3  >  0. 

Since  by  definition  C  >  q^ ,  we  have  verified  (3.7)  and  completed  the  proof.  □ 


Remark  3.3  Let  tis  also  note  that  under  the  assumptions  of  Theorem  3.2  for  the  exact 
greedy  algorithm,  (1.4)  gives 

<  CMn2ne~ana.  (3.8) 

This  latter  estimate  is  asymptotically  sharper  than  (3.4)  in  Theorem  3.2  as  long  as  a  >  1, 
whereas  (3.4)  is  asymptotically  sharper  for  a  <  1.  For  a  =  1,  (3.8)  is  asymptotically  sharper 
provided  a  >  In  2,  and  not  if  a  <  In  2. 

3.3  Robustness 

In  a  PDE  context,  for  example  as  described  by  (1.1),  the  reduced  basis  functions  fj  cannot 
be  computed  exactly.  Instead,  one  relies  on  a  space  discretization,  such  as  by  finite  elements, 
in  order  to  approximate  them  by  functions  fj  with  some  prescribed  accuracy.  This  leads  to 
a  modification  of  the  spaces  Fn  generated  by  the  weak  greedy  algorithm  that  we  shall  now 
describe  and  analyze. 

We  denote  by  £  >  0  the  accuracy  to  which  a  given  element  /  £  hF  is  computed  when  it 
is  chosen  as  a  member  of  the  reduced  basis.  At  the  first  step  of  the  weak  greedy  algorithm 
we  determine  /o  £  T  such  that  ||/o||  >  ysup^gjp- 1|/||.  However,  rather  than  receiving  f0  we 
receive  the  noisy  version  /o  which  is  not  even  necessarily  in  T .  All  we  know  about  /o  is  that 

II /o  -  /oil  <  e. 

This  means  that  our  first  space  is  now  F\  :=  span{/0}  and  it  is  used  for  the  selection  of  f\. 
In  general,  if  /0, . .  . ,  fn-i  have  been  found  and  Fn  :=  span {/0, . .  . ,  fn- 1}  then  we  determine 
fn  such  that  dist(/n,Fn)  >  7sup^6^-dist(/,  Fn).  Rather  than  fn  we  receive  the  noisy  version 
fn  for  which  we  only  know  ||/n  —  /n||  <  e.  Let  us  define 

dn(JF)  :=  supdist(/,Fn), 
feT 

which  is  the  performance  of  this  noisy-weak  greedy  algorithm  on  T . 

The  following  shows  that  the  result  of  Theorem  3.1  is  robust  with  respect  to  this  pertur¬ 
bation. 


10 


Theorem  3.4  Suppose  that  do(F)  <  M  and 


dn(F)  <  Mn~a,  n  >  0,  (3.9) 

for  some  M,  a  >  0.  Then, 

an(F)  <  Cm&x{Mn~a,e},  n  >  0,  (3.10) 

with  C  :=  (1  +  2 a)C(a,  %),  where  C(a,  |)  is  the  constant  of  Theorem  3.1  for  a  and 

Proof:  Let  N  be  the  largest  integer  such  that  MN~a  >  e.  We  shall  first  prove 

an(F)  <  C  rna x{Mn~a,  e},  1  <  n  <  N  +  1.  (3-11) 

Fix  a  value  of  n  £  {1, . . . ,  IV}.  If  dn(F)  <  Ce,  then  we  have  verihed  (3.11)  for  this  value  of 
n,  as  well  as  for  the  next  ones.  So  we  can  assume  &n(F)  >  Ce.  This  implies  that  &j(F)  >  Ce 
for  all  j  —  1,  •  •  •  ,  n. 

We  define  Tn  :=  JFu{/0, . . . ,  fn-i}  and  first  examine  the  Kolmogorov  widths  of  Tn.  Since 
each  of  the  fj  are  within  e  of  an  element  from  T ,  using  (3.9)  we  have  for  any  1  <  m  <  2A^, 

dm(Fn)  <  Mm~a  +  e  <  (1  +  2 a)Mm~a. 

For  m  >  2 N,  using  the  fact  that  Tn  has  been  obtained  by  adding  n  functions  to  T ,  we  have 
dm{Tn)  <  dm-niJ7)  <  M(m  -  n)~a  <  M{m/2)~a  <  2 aMm~a. 


In  summary, 


dm(Fn)  <  (1  +  2 a)Mm  m  >  0. 


Next,  we  examine  the  weak  greedy  algorithm  applied  to  Tn  with  the  weakness  parameter 
t).  We  claim  that  the  functions  /0, . . . ,  fn  could  be  chosen  in  this  order  by  this  algorithm. 
First,  we  define  F0  :=  {0},  and  therefore  for  any  /,  dist(/,  F0)  =  ||/||.  Observe  that  for 
m  —  0, . . . ,  n, 


dist(/m,  Fm)  >  dist(/m,  Fm)  -  £  >  7  sup  dist (/,  Fm)  -  £  >  7  sup  dist (/,  Fm)  -  2e, 

/GJ7  /G^„ 


and  since  sup dist (/,  Fm)  >  dm(F)  >  Ce  and  C  =  (1  +  2a)C(a,  |)  >  4y  \  we  also  have 


e  <  —  sup  dist  (jT,  Fn 
4 


Therefore, 

dist  (fm,Fm)  > 'd-  sup  dist(/,Fm). 
z  fern 

We  have  thus  verihed  that  /0, . . . ,  fn  is  an  allowable  sequence  to  choose  under  the  hypothesis 
of  Theorem  3.1  for  the  set  Fn  and  the  parameters  a  and  f.  If  we  apply  the  theorem,  we 
thus  obtain  that  for  m  >  0, 

am{Fn)  <  C(a,  |)(1  +  2a)Mm~a  <  CMm~a. 
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Since  /0, . . . ,  fn  is  an  allowable  sequence, 

dm{F)  =  supdist(/,  Fm)  <  am(Fn),  1  <  m  <  n  +  1. 

feT 

Thus,  we  have  proved  (3.11)  for  1  <  n  <  N  +  1.  Finally,  if  n  >  N  +  1,  then 

an(F)  <  aN+1(F)  <  CM(N  +  l)~a  <  Ce. 

This  shows  that  (3.10)  holds  for  n  >  N  as  well.  □ 

By  similar  arguments,  one  obtains  the  following  robust  version  of  Theorem  3.2. 

Theorem  3.5  Suppose  that 


dn{F)  <  Me~ana,  n>  0, 

for  some  M,  a,  a  >  0.  Then,  setting  /3  :=  one  has 

<  C  max{Me_cn/3,  e},  n  >  0, 

where  c,C  are  constants  that  depend  on  a,  a  and  on  the  weakness  parameter  7 . 

We  do  not  provide  the  proof  which  is  similar  to  the  proof  of  Theorem  3.4. 

4  Direct  comparison  between  and  d^J7) 

The  results  of  the  previous  section  show  that  estimates  on  the  rate  of  decay  of  crn{T)  may 
be  derived  from  estimates  on  the  rate  of  decay  of  dn(!F),  but  they  do  not  provide  a  direct 
comparison  between  the  individual  quantities  <Jn(jF)  and  dn(^r)-  In  this  section,  we  discuss 
such  comparison  and  show,  in  particular,  that  estimates  of  the  form  (Jn{T)  <  Cdn(tF)  cannot 
be  met. 

4.1  Good  approximating  subspaces 

The  optimal  subspace  for  the  Kolmogorov  width  dn(!F)  is  not  necessarily  generated  by  ele¬ 
ments  from  T .  However,  we  are  considering  only  subspaces  of  the  latter  type  in  our  approx¬ 
imation  algorithms.  Thus,  it  is  of  interest  to  understand  how  well  we  can  approximate  T 
when  using  spaces  Y  of  dimension  n,  where  Y  is  spanned  by  n  elements  from  T .  That  is, 
Y  =  span{g!, . . . ,  gn},  where  g*  G  T ,  i  =  1, . . . ,  n.  We  define  Wfn(jF)  to  be  the  collection  of 
all  such  subspaces  Y  and  set 

dJF)  '■=  inf  sup  dist (f,Y). 

Y&Mn{F) 

Clearly  Jn(jF)  >  dn(F).  The  following  result  draws  a  comparison  between  dn  and  dn. 
Theorem  4.1  The  following  holds: 
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(i)  For  any  compact  set  P  and  any  n  >  0,  we  have  dn(P)  <  (n  +  1  )dn(P). 

(ii)  Given  any  n  >  0  and  e  >  0,  there  is  a  set  P  such  that  dn(P)  >  {n  —  1  —  e)dn(P). 

Proof:  To  see  (i),  let  Y  be  the  optimal  Kolmogorov  space  for  P  of  dimension  n.  We  let 
0i, . . . ,  (j)n  be  an  orthonormal  basis  for  Y  and  let  P  be  the  projection  onto  Y.  Therefore,  for 
any  f  E  P, 

n 

pf  = 

3= 1 

and  ||/  —  Pf  ||  <  dn(P),  f  E  P .  For  any  {/i, . . . ,  fn}  C  P,  we  consider  the  determinant 
D(fi, . . . ,  fn)  :=  det((/j,  <f>j))  and  choose  fi, . . . ,  fn  to  maximize  the  absolute  value  of  this 
determinant.  For  any  f  E  P,  we  can  write  Pf  =  Y^i=i  aiPfi->  where 

D(Pfl,  •  •  •  ,  Pfi-l,  Pf ,  Pfi+l,  •  •  •  ,  Pfn)  D(fl,  ■  ■  ■  ,  fi-1,  f,  fi+ 1, 
D(Pf1,...,Pfn)  D(fi, . . . ,  /„) 

because  P  is  an  orthogonal  projector.  From  the  definition  of  /i, . . . ,  fn ,  we  have  that  \af  <  1. 
Now,  we  can  write 

n  n 

f  -^2aifi  =  f  -  pf +Y1  a^pfi  - 

i=  1  i=l 

It  follows  that 

n 

11/ oiJiW  <  (n+  1  )dn{P), 

i=  1 

as  claimed. 

As  for  (ii)  let  /i  >  0  and  P  =  {/i,  /2,  -  -  - ,  fn,  fn+ 1,  0, .  . . ,  0, . . .  },  where  ./)■  :=  +  /xen+1, 

.7  =  1,  ■  ■  ■ ,  n,  and  /n+1  :=  YTj=i  ej  +  ^n+i-  By  choosing  K*  :=  span{e1; . . . ,  enj,  we  see 
that  dn(P)  <  fi.  Now  consider  any  n  dimensional  space  Y  =  span{^!, . . . ,  gn}  with  the 
Qj  E  P .  If  fn+ 1  is  not  one  of  the  gv  then  the  only  choice  of  Y  is  Y  =  span{/i , . . . ,  fn}. 
For  this  choice  of  Y,  the  vector  y  :=  —  en+i  +  /i  YT3=\  ej  is  orthogonal  to  Y  and  has 

norm  ||?/||  =  \J  1  +  np2.  So  the  error  in  approximating  fn+ 1  by  the  elements  of  Y  is 
(1  +  np2)~LP(y,  fn+i)  =  (1  +  n/r2)'1/2(n  —  1  )p.  On  the  other  hand,  if  fn+\  is  included  as 
one  of  the  gi, . . . ,  gn ,  then  one  of  the  fj,  j  —  1, . .  . ,  n,  is  not  included.  Without  loss  of  gen¬ 
erality,  we  can  assume  that  this  is  f\.  Now,  we  can  take  y  :=  (n  —  2)/rei  —  /i  YTj=2  ej  +  en+i 
which  has  norm  \Jl  +  y?(n2  —  3n  +  3).  This  gives  that  the  error  in  approximating  f\  is 
(n  —  l)/i(l  +  /i2(n2  —  3 n  +  3))-1//2.  So  for  either  of  these  two  cases,  if  we  take  y  sufficiently 
small  we  obtain  (ii).  □ 


Remark  4.2  Since  an(P )  >  dn(P),  an  immediate  consequence  of  the  second  item  in  the 
above  theorem  is  that  given  any  n  >  0  and  e  >  0,  there  is  a  set  P  such  that  an(P)  > 
(n  —  1  —  e)dn(P).  In  particular,  this  shows  that  one  cannot  expect  a  general  estimate  of  the 
form  crn(P)  <  Cdn(P).  Note  also  that  there  is  no  contradiction  between  this  second  item 
and  the  result  of  Theorem  3.1  concerning  polynomial  rate  of  decay. 
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4.2  An  improvement  of  (1.4) 

We  shall  now  show  that  (1.4)  can  be  improved  by  replacing  the  factor  n2n  by  2n.  Here,  we 
work  in  the  specific  setting  of  the  exact  greedy  algorithm  corresponding  to  7  =  1,  and  use  the 
matrix  formulation  of  the  greedy  algorithm  introduced  in  §2.  The  main  vehicle  for  proving 
this  upper  estimate  is  to  understand  how  well  the  coordinate  vectors  e*  can  be  represented 
as  a  linear  combination  of  the  fj,j  =  0, 1, ,  i.  This  information  is  provided  by  the  inverse 
of  the  matrix  A.  We  shall  need  the  following  lemma  reflecting  this  fact. 

Lemma  4.3  For  each  n  >  1,  we  can  write 

n 

®nAn  ^  ^  bn ( j )  fj ,  (4-1) 

3=0 

where 

n 

j)I<2”.  (4.2) 

3=0 

Proof:  Bounds  of  this  type  were  given  in  [1],  For  completeness,  we  repeat  the  simple  proof. 
Let  bn  :  =  (6n(j))"=0  and  An  be  the  submatrix  of  A  formed  by  all  entries  Ojj,  0  <  i,  j  <  n. 
The  7-th  row  («yj;)"=o  of  A  can  be  identified  with  fj  and  (4.1)  can  be  then  rephrased  as 

n  ^n&n- 

Since  this  is  an  upper  triangular  system,  it  can  be  solved  by  back  substitution  which  yields 
bn{n)  =  1  and  for  j  =  n  —  1, . . . ,  0, 


°jK{j)  =  -  ^  bn(i)aitj.  (4.3) 

i=j+ 1 

From  P2,  we  have  \atj\  <  ctj,  i  >  j  +  1.  Using  this  in  (4.3)  gives 

n 

Mi)  I  <  Kii)  |- 

i=j+ 1 

Since  bn(n )  =  1,  we  obtain  \bn(n  —  1)|  <1  and  in  general  \bn(n  —  j)\  <  2J_1,  1  <  j  <  n.  We 
obtain  (4.2)  by  adding  these  inequalities.  □ 

Theorem  4.4  Let  T  be  an  arbitrary  compact  set  in  a  Hilbert  space  74.  For  each  n  =  1,2,... 
we  have 

2«+i 

<?n{F)  <  (4.4) 

Proof.  Let  Y  be  a  space  of  dimension  n  which  attains  dri(i-’).  Then,  in  particular,  we  have 

dist(./j,  Y)  <  dn(Jr),  j  —  0, . . .  ,n.  (4.5) 
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Since  the  fc-th  coordinate  of  each  of  the  vectors  fj,  j  =  0, . . .  ,n,  is  zero  for  k  >  n  +  1, 
we  can  achieve  (4.5)  for  a  space  Y  of  dimension  n  contained  in  the  span{eo, . . .  ,en}.  By 
(4.5),  we  can  find  for  each  j  —  0, . . . ,  n,  a  yj  e  Y  such  that  | 1  j)  —  yj ||  <  dn(JF).  Setting 
Vi  :  =  Y^k=o^i(k)yk  £  Y ■>  it  follows  from  Lemma  4.3  that 

dist(ajej,  Y)  =  Wcr^i  -  Vi\\  <  2*dn(J'),  i  =  0, 1 . . . ,  n. 


Now  Y  can  be  characterized  by  a  unit  vector  y  =  0), . . . ,  y(n))  e  span{eo, . . . ,  en}  which 

is  orthogonal  to  Y.  For  this  y,  we  have  from  (4.2)  that 


crn\y{})\  <  =  \{<7iei,y)\  =  \  (<?&  -  vhy)\  <  ||  -  Ui||  <  2ldn(Jr),  i  =  0,...,n. 

Thus, 


°2n  =  <%Y^\y(i)\2  <  221 


< 


2^12+2  _ 


2 2n+2 


i= 0 


i= 0 


3  dn(Tf  <  —dn(F)2, 


which  completes  the  proof  of  the  theorem. 


□ 


For  any  n  >  0,  we  finally  present  an  example  of  a  set  T  =  Tn  such  that  for  the  exact 
greedy  algorithm 

o‘n(^r)  >  (1  —  e)2 ndn[T)  for  all  e  >  0, 

which  reveals  that  Theorem  4.4  is  sharp  safe  for  the  constant  2/\/3.  An  example  with  a 
similar  type  of  implication  was  also  given  in  [1],  In  our  example,  the  lower  triangular  matrix 
A,  defined  in  §2,  has  the  following  form:  the  j-  th  column  has  a j  in  position  ( j,j )  and  all 
entries  in  positions  (i,j)  with  0  <  j  <  i  <  n  will  equal  a  fixed  constant  aj.  All  rows  of  A 
with  indices  i  >  n  are  identically  zero. 

Let  t0  =  1  ■■■An  be  positive  real  numbers.  We  define 


(To  1,  &k+ 1 


TT  2-y/fjtj  +  l 

f=o  tJ  +  ^+i 


tk  +  tk+ 1 


k  —  0, . . . ,  n  -*  1. 


Since  each  factor  in  the  product  is  less  or  equal  to  1,  the  sequence  ay  is  non-increasing. 

Furthermore,  we  define  cy  :=  oj— - T+i  These  values  for  cy  serve  to  define  the  matrix  A 

tk  +  tk+l 

as  described  above.  We  note  that  cf\  =  a\  +  crjji+1,  k  —  0, . . . ,  n  —  1,  from  which  we  derive 


cq 


k- 1 

^a?  =  l,  k  =  0,.. 

3=0 


n. 


(4.6) 


We  define  fk  as  the  fc-th  row  of  A  and  T  =  {/o,  /i,  ji,  ■  ■  ■  }•  Thus,  A  satisfies  properties 
PI  and  P2  and,  according  to  Remark  2.1,  it  follows  that  A  is  a  matrix  that  represents  a 
possible  scenario  of  the  weak  greedy  algorithm  applied  to  T .  In  fact,  since  djj  =  <jj,  this 
scenario  corresponds  to  the  particular  case  of  the  exact  greedy  algorithm.  Note  also  that  by 
(4.6)  each  fk  has  norm  one. 

We  want  now  to  show  that  with  a  suitable  choice  of  the  tk,  the  Kolmogorov  width 
dn(tF)  will  be  quite  small.  For  this,  it  will  be  enough  to  construct  a  unit  length  vector 
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y  =  E^Lo  V  (j )  eJ  ■>  whose  inner  product  with  each  of  the  fk,  k  =  0,  is  small.  We 

define  £  :=  ?==  and  y(j)  :=  Sy/t]  which  gives  that  ||r/||  =  1.  We  first  note  that 


(/o,  y)  =  a0ey/%  =  e.  We  can  compute  the  other  inner  products  of  y  with  the  fk  by  using 
the  following  equality 


{fk+ liV)  —  °~k+i£\/tk+i  + 


3=0 


km 1 


&k+l  £\ftk+l  +  Ofc^'s/E  +  ^2aj£y/tj 

3=0 


=  (yke  \  yft 


<Jk+ 1 ' 


"2 \ftktk+ 


-  +  Vh 


tk  tk+1  } 


fc-1 


=  CTfc£- 


tk  +  tk+1 

2tfc-|-l\/^fc  T"  tky/tf  ^fc+l\/^fc 
tk  +  ^fc+1 


r + 

>  j= 0 


tk  +  ^fc+1 

fc-1 

Oj£  y/tj 

i=0 


fc-1 

—  CTfegyE;  +  ^  "  aj£\/tj 

3=0 

=  (fk,y),  k  —  0, . . .  ,n  —  1. 

Thus  (fk,  y)  —  £  for  k  —  0, n.  If  we  take  Y  as  the  linear  space  contained  in  span{e0, . . . ,  en } 
which  is  orthogonal  to  y,  then  Y  has  dimension  n  and  therefore 

dn(F)  <  e- 

As  a  particular  choice,  we  now  choose  tk  :  =  rk ,  k  =  0 where  r  is  large.  Then, 
one  checks  that  an  =  2nr~nt2(l  +  l/r)~n.  In  addition,  I  =  j=o ^  =  y^E-T'  —  rnt2. 
Therefore, 

fjS-  >  2"(1  +  l/r)-". 

dn\3~) 

If  we  let  r  — >  oo  with  n  fixed,  we  see  that  in  general  we  cannot  have  an  estimate  better  than 
that  given  in  (4.4)  save  for  the  factor  \J 4/3. 
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